1. Field of the Invention
The present invention pertains to a solid-state imaging apparatus for motion detection which detects differences between frames of images. More specifically, the present invention pertains to an imaging apparatus for motion detection that compares a pixel output of pixels in an image frame to determine whether objects within the image move and to provide for error correction of false motion signals.
2. Description of the Related Art
Prior art processing apparatuses for motion detection sequentially transfer image data from a solid-state imaging apparatus and detect motion based on differences between frames of this image data.
FIG. 11 is representative of a prior art image processing apparatus for motion detection 100. The image processing apparatus for motion detection 100 consists of a solid-state imaging apparatus 101, an A/D conversion circuit 102 that converts the analog image signal from the solid-state imaging apparatus 101 into a digital signal, a first image memory 103 and a second image memory 104 that save digital signals from A/D conversion circuit 102, and an image processing circuit 105 that compares the digital image data saved in the first and second image memories 103, 104 against one another to detect motion.
In this motion detection apparatus 100, a first frame image signal is first converted into a digital signal by the A/D conversion circuit 102, and then saved in first image memory 103.
Next a second frame image signal is also converted into a digital signal by the A/D conversion circuit 102 and saved in second image memory 104.
The image processing circuit 105 then compares pixels of the digital signal saved in the first image memory 103 with pixels of the digital signal saved in the second image memory 104. The processing circuit detects pixels that differ by more than a specified threshold value and generates a signal indicating detection of a moving object (hereafter the xe2x80x9cmoving object signalxe2x80x9d).
In this manner, comparison of successive frames permits detection of motion of a subject. Nevertheless, the aforesaid conventional image processing apparatus for motion detection 100 is not ideal. The motion detection circuitry for the solid-state imaging apparatus 101 is complicated making the image processing apparatus for motion detection 100 overly large and costly.
Another defect of the prior art is that the image signal output from solid-state imaging apparatus 101 is an analog signal, which is supplied to A/D conversion circuit 102. Therefore, the analog signal is conducted along a path presenting an opportunity to be easily affected by noise (interference), which causes the image processing circuit to erroneously generate the moving object signal.
Furthermore, in the motion detector apparatus 100, the dynamic range of the analog image signal is limited by the A/D conversion circuit 102. The input dynamic range of A/D conversion circuit 102 is usually narrower than the dynamic range of the solid-state imaging apparatus 101. Therefore, there is a defect in that the wide dynamic range of solid-state imaging apparatus 101 cannot be effectively used in the course of detecting and processing a moving object.
Also, A/D conversion circuit 102 has a sample rate that may become out of phase with the successive frames provided by the imaging apparatus 101. This type of phase shifting in inter-frame sampling timing can create a slight phase shift in the pixel position to be compared at the image processing circuit 105. If this type of phase shift occurs, a stationary body may have inter-frame differences at its edge portions. Therefore, prior art solid-state imaging apparatuses do not provide the desired precision and reliability of moving object detection.
One proposal for avoiding these defects is to provide a memory to store the image signal for the immediately previous frame and the current frame in each pixel of solid-state imaging apparatus 101, and to additionally provide each pixel with a comparison circuit to compare the image signal stored in this memory, and to generate a moving object signal for each pixel.
However, this design makes the structure of the unit pixel complicated, and reduces the numerical aperture and resolution of the solid-state imaging apparatus 101. In addition, this design can output only the moving object signal from each pixel. Thus, this design can not simultaneously provide an image signal and a motion signal.
It is generally known that a solid-state imaging apparatus comprising a semiconductor device experiences charge fluctuations, which create shot noise. The magnitude of shot noise is proportional to the square root of the signal magnitude. Therefore, the brighter the subject and the higher the signal level, the greater the shot noise that is created.
As a result, in bright locations shot noise looms large in inter-frame differences. If shot noise occurs in inter-frame differences and exceeds the threshold value for a moving-object decision, erroneous motion detection may occur.
One proposal for avoiding erroneous detection due to shot noise is to set the comparison threshold value for differences between frames uniformly high. Nevertheless, this sort of countermeasure has the problem that sufficient motion detection cannot be performed for a low-contrast subject.
Another known problem of using a semiconductor imaging apparatus is that incorrect motion detection may occur when the field is extremely bright or extremely dark because motion signals can not be generated accurately. Also, in addition to the case described above, background differences between frames also occur, such as when tree leaves wave in a wind. This sort of motion is small motion in the background, and should be distinguished from motion of the intended subject that is being monitored.
The imaging apparatus for motion detection of the present invention includes an imaging unit that receives incident light on an array of pixel elements that provide a pixel output corresponding to the incident light thereby generating an output signal. The imaging apparatus compares sequential pixel output signals from a single pixel, and generates a motion signal which indicates in pixel units whether or not there is a change within the field of coverage. A motion signal processing circuit sequentially fetches the image signals and motion signals associated with the same pixel that were generated by the imaging unit, and determines whether to externally output the motion signals based on the image signals.
Because image signals and motion signals are generated simultaneously, and the motion signals an be controlled based on the image signals, various types of signal processing of the motion signal can be performed easily. Consequently, additional functions relating to the motion detection of an object in the field can be easily implemented.
For example, if the brightness level, or the color components of the desired subjects, or if the brightness level or the color components of objects that are to be excluded as subjects for motion detection are known, it is possible to determine whether or not to externally output a motion signal according to the brightness level and color components of the image signal. As a result, it is possible to reliably detect the motion of the desired subjects only.
Thus with an imaging apparatus for motion detection which compares the pixel output that is being continuously output from the same pixel and thereby generates a motion signal, if any noise is superimposed on an image signal corresponding to the pixel output, the effect of the noise will also appear in the motion signal, and stationary objects can be mistakenly detected as moving objects. However, because the motion signal can be disabled, any noise that is superimposed on the image signal can be reliably reduced, and the motion of objects in the field can be accurately detected.
In a preferred embodiment, the motion signal processing circuit externally outputs those motion signals generated by the imaging unit corresponding to pixel elements in which the image signal has a brightness level that is outside a prescribed range, and externally outputs a signal indicating that there is no change in the field associated with pixel elements in which the image signal has a brightness level that is within the prescribed range. The brightness level of an image signal may be outside a prescribed range by exceeding a prescribed upper limit or by falling below a prescribed lower limit.
Specifically, the motion signal can be disabled when shot noise becomes superimposed on an image signal, as well as under conditions in which it is susceptible to the effects of random noise. Consequently, erroneous detection of stationary objects as moving objects caused by noise can be reliably reduced, and the motion of objects in the image field can be accurately detected.
In preferred embodiments, the imaging unit includes a plurality of photoreceptors, arranged in an array, that generate the pixel output according to the incident light. A plurality of vertical read lines are coupled to columns of the photoreceptors and a vertical transfer circuit that sequentially selects a row of the photoreceptors and sequentially outputs to the vertical read lines the previous frame""s pixel output and then the current frame""s pixel output. A comparison circuit is coupled to each of the vertical read lines and compares the previous frame""s pixel output and the current frame""s pixel output that are sequentially transferred via the vertical read lines. A horizontal transfer circuit horizontally transfers the comparison circuit""s comparison results that are output on each of the vertical read lines. An image signal output circuit selectively fetches and horizontally transfers either the previous frame""s pixel output or the current frame""s pixel output which are sequentially transferred via the vertical read lines.
The references to horizontal and vertical transfer of signals is a convenient reference to the orientation of the array of photoreceptors as present in the accompanying drawings of the invention. In this context, a vertical transfer refers to a transfer of signals from a particular row of the array to respective output lines and a horizontal transfer refers to a sequential transfer of signals from the array row on the output line to an output terminal. Alternatively, the vertical transfer may apply to a transfer of signals from a column of the array and the horizontal transfer may refer to a transfer of the signals from the array column to the output terminal.
The present invention also provides a solid-state imaging apparatus for motion detection which reduces external image comparison processing circuits and discriminates against shot noise and small background motions when detecting motion of an intended subject. In addition, the present invention provides a solid-state imaging apparatus for motion detection which can simultaneously output a moving object signal and an image signal.
The present invention reduces erroneous detection of motion in the screen""s horizontal and vertical directions. In addition, the present invention can reduce erroneous detection of motion in the time axis direction.
The term xe2x80x9cframexe2x80x9d in this application refers to a set of pixel signals from the array of pixel elements. Accordingly, the solid-state imaging apparatus for motion detection need not be limited to devices which perform progressive scanning. For example, the present invention may be applied to devices that perform interlaced scanning.
In the present invention, the image signal output operation does not monopolize the vertical read line, so it does not interfere with the operation of the motion detection side. Accordingly, it is possible to output a moving object signal and an image signal simultaneously.
Embodiments of the present invention also provide a level decision circuit that decides the level of an image signal output from an image signal output circuit, and an output switching circuit that switches and outputs the output of the logical calculation circuit and the comparison circuit""s comparison results according to the level decision circuit""s decision result.
Because shot noise occurs in proportion to the square root of the signal level, it is concentrated in the high luminance areas of the image signal. Therefore, by deciding whether the image signal exceeds a prescribed level, the level decision circuit can determine the regions most likely to include significant shot noise.
Therefore, for example, when the image signal exceeds a prescribed level the output switching circuit should selectively output the logical calculation circuit output, and when the image signal falls below a prescribed level the output switching circuit should output the comparison circuit""s comparison result. This sort of switching operation can selectively and reliably reduce erroneous detection of motion caused by shot noise. Additionally, there is no unnecessary removal of isolated regions with little shot noise, so the detection apparatus can reliably detect the motion of small detection subjects.
On the other hand, in cases in which the signal level is extremely small, random noise from circuit systems and the like dominate. (In particular, random noise is amplified and strongly manifested in connection with signal level drops when a peak AGC circuit or the like is interposed in a circuit.)
Performing output switching between the logical calculation circuit output and the comparison circuit""s comparison result in response to the level decision circuit""s decision result as in the examples given above makes it possible to selectively reduce motion detection errors due to noise, etc., and makes it possible to detect the motion of small detection subjects.
The present invention also provides a solid-state imaging apparatus for motion detection wherein the comparison circuit is a circuit that decides whether or not the current frame""s pixel output and the previous frame""s pixel output agree within an allowed range, and that outputs a binary signal according to the truth or falsity of the decision result.